A Note on the Theme of Too Many Instruments

The “difference” and “system” generalized method of moments (GMM) estimators for dynamic panel models are growing steadily in popularity. The estimators are designed for panels with short time dimensions (T), and by default they generate instruments sets whose number grows quadratically in T. The dangers associated with having many instruments relative to observations are documented in the applied literature. The instruments can overfit endogenous variables, failing to expunge their endogenous components and biasing coefficient estimates. Meanwhile they can vitiate the Hansen J test for joint validity of those instruments, as well as the difference-in-Sargan/Hansen test for subsets of instruments. The weakness of these specification tests is a particular concern for system GMM, whose distinctive instruments are only valid under a non-trivial assumption. Judging by current practice, many researchers do not fully appreciate that popular implementations of these estimators can by default generate results that simultaneously are invalid yet appear valid. The potential for type I errors—false positives—is therefore substantial, especially after amplification by publication bias. This paper explains the risks and illustrates them with reference to two early applications of the estimators to economic growth, Forbes (2000) on income inequality and Levine, Loayza, and Beck (LLB, 2000) on financial sector development. Endogenous causation proves hard to rule out in both papers. Going forward, for results from these GMM estimators to be credible, researchers must report the instrument count and aggressively test estimates and specification test results for robustness to reductions in that count.dynamic panel estimation, difference GMM, system GMM, Stata, Arellano-Bond, Blundell-Bond, generalized method of moments, autocorrelation, finance and growth, inequality and growth

Aid Project Proliferation and Absorptive Capacity

Much public discussion about foreign aid has focused on whether and how to increase its quantity. But recently aid quality has come to the fore, by which is meant the effectiveness of the aid delivery process. This paper focuses on one process problem, the proliferation of aid projects and the associated administrative burden for recipients. It models aid delivery as a set of production activities (projects) with two inputs, the donor’s aid and a recipient-side resource, and two outputs, namely, development and “throughput,” which proxies for the private benefits for both donor and recipient of implementing projects, from kickbacks to career rewards for disbursing. The donor’s allocation of aid across projects is taken as exogenous while the recipient’s allocation of its resource is modeled and subject to a budget constraint. Unless the recipient cares purely about development, increasing aid can reduce development in some circumstances. Sunk costs, representing the administrative burden for the recipient of donor meetings and reports, are introduced. Using data on the distribution of projects by size and country, simulations of aid increases are run in order to examine how the project distribution evolves, how the recipient’s resource allocation responds, and how this affects development if the recipient is not a pure development optimizer. With Cobb-Douglas production, a threshold is revealed beyond which marginal aid effectiveness drops sharply. It occurs when development maximization calls for the recipient to withdraw from some donor-backed projects—but the recipient does not, for the sake of throughput. Donors can push back this threshold by moving to larger projects if there are scale economies in aid projects.Foreign aid, donor coordination, project proliferation, absorptive capacity

The Anarchy of Numbers: Aid, Development, and Cross-country Empirics

Recent literature contains many stories of how foreign aid affects economic growth. All the stories hinge on the statistical significance in cross-country regressions of a quadratic term involving aid. Among the stories are that aid raises growth (on average) 1) in countries where economic policies are good; 2) in countries where policies are good and a civil war recently ended; 3) in all countries, but with diminishing returns; 4) in countries outside the tropics; 5) in countries with difficult economic environments, characterized by declining or volatile terms of trade, natural disasters, or low population; or 6) when aid increases in countries experiencing negative export price shocks. The diversity of results prima facie suggests that many are fragile. Easterly et al. (2004) find the aid-policy story (Burnside and Dollar, 2000) to be fragile in the face of an expansion of the data set in years and countries. The present study expands that analysis by applying more tests, and to more studies. Each test involves altering just one aspect of the regressions. All 19 tests are derived from sources of variation that are minimally arbitrary. Twelve derive from specification differences between studies, what Leamer (1983) calls “whimsy.” Three derive from doubts about the appropriateness of the definition of one variable in one study. The remaining four derive from the passage of time, which allows sample expansion. This design allows an examination of the role of “whimsy” in the results that are tested while minimizing “whimsy” in the testing itself. Among the stories examined, the aid-policy link proves weakest, while the aid-tropics link is most robust.foreign aid, economic growth, robustness testing

Through the Looking-Glass, and What OLS Found There: On Growth, Foreign Aid, and Reverse Causality

The cross-country literature on foreign aid effectiveness has relied on the use of instruments to distinguish causality from mere correlation. This paper uses simple non-instrumental techniques in the spirit of Granger to demonstrate that the main aid-growth connection is a negative causal relationship from growth to aid—-aid, that is, as a fraction of recipient GDP. Coarsely, when GDP goes up, aid/GDP goes down. The endogeneity of aid, long suspected, is real. Less understood is that adding certain common controls to regressions puts this relationship through the looking glass, flipping both its sign and apparent direction: aid seems to cause growth. Ideally, instrumentation expunges the endogeneity shown here. In practice, estimates of aid’s impact have run into problems. Autocorrelation in the errors is widespread, and can render endogenous lagged variables used as regressors or instruments. The pitfalls of “difference” and “system” include invalidity and proliferation of instruments. Multicollinearity in term pairs of interest, such as aid and aid2 or “project” and “program” aid, can amplify endogeneity bias. The combination of specification problems and widespread fragility (shown in earlier work) leads to pessimism about the ability of cross-country econometrics to demonstrate aid effectiveness. This does not rule an average positive effect, nor does it contradict the fact that aid has saved millions of lives, but it does suggest that the average effect on economic growth is too small to be detected statistically.foreign aid, economic growth

How Do the BRICs Stack Up? Adding Brazil, Russia,India, and China to the Environment Component of the Commitment to Development Index

The Commitment to Development Index (CDI) ranks 21 of the world’s richest countries on their dedication to policies that benefit the five billion people living in poorer nations. Moving beyond simple comparisons of foreign aid, the CDI ranks countries on seven themes: quantity and quality of foreign aid, openness to developing-country exports, policies that influence investment, migration policies, stewardship of the global environment, security policies and support for creation and dissemination of new technologies. This year for the first time, CGD research fellow David Roodman extended the environment component of the Index to cover four of the biggest developing countries: Brazil, Russia, India and China, a group Goldman Sachs dubbed the “BRICs.” This working paper explores the indicators that make up the environment component (global climate, sustainable fisheries, and biodiversity and global ecosystems) and explains how the BRIC countries stack up to their right-country counterparts. He finds that the BRICs score remarkably well compared to the 21 rich countries covered by the Index: when thrown in with the usual 21, they rank second, fourth, fifth, and eleventh. They generally perform well on the greenhouse gas emissions, consumption of ozone-depleting substances, and tropical timber imports. And the BRICs have joined important international environmental accords. As a group, their major weakness is low gas taxes. In addition, Amazon deforestation and heavy fossil fuel use pull Brazil and Russia, respectively, below the CDI 21 average on greenhouse emissions per capita. China’s abstention from the U.N. fisheries agreement puts it a half point below the other BRICs.environment, Commitment to Development Index (CDI)

How to Do xtabond2: An Introduction to "Difference" and "System" GMM in Stata

The Arellano-Bond (1991) and Arellano-Bover (1995)/Blundell-Bond (1998) linear generalized method of moments (GMM) estimators are increasingly popular. Both are general estimators designed for situations with "small T, large N" panels, meaning few time periods and many individuals; with independent variables that are not strictly exogenous, meaning correlated with past and possibly current realizations of the error; with fixed effects; and with heteroskedasticity and autocorrelation within individuals. This pedagogic paper first introduces linear GMM. Then it shows how limited time span and the potential for fixed effects and endogenous regressors drive the design of the estimators of interest, offering Stata-based examples along the way. Next it shows how to apply these estimators with xtabond2. It also explains how to perform the Arellano-Bond test for autocorrelation in a panel after other Stata commands, using abar. The paper closes with some tips for proper use.dynamic panel estimation, difference GMM, system GMM, Stata, Arellano-Bond, Blundell-Bond, generalized method of moments, autocorrelation

Macro Aid Effectiveness Research: A Guide for the Perplexed

Like many public policy debates, that over whether foreign aid works carries on in two worlds. Within the research world, it plays out in the form of papers full of technical language, formulas, and numbers. Outside, the arguments are plainer and the audience broader, but those academic studies remain a touchstone. While avoiding jargon, this paper reviews recent, contending studies of how much foreign aid affects country-level outcomes such as economic growth and school attendance rates. This particular kind of study is ambitious: it is far easier to evaluate a school-building project, say, on whether the school was built and children filled its seats than to determine whether all aid, or large subcomponents of it, made the economy grow faster. Because of its ambition, this literature has attracted attention from those hoping for clear answers on whether aid "works.' On balance, the quantitative approach to exploring grand questions about aid effectiveness, which began 40 years ago, was worth trying and is probably worth pursuing somewhat further. But the literature will probably continue to disappoint as often as it offers hope. Perhaps the biggest challenge is going beyond documenting correlations to demonstrating causation—not just that aid went hand-in-hand with economic growth, but caused it. Aid has eradicated diseases, prevented famines, and done many other good things. But given the limited and noisy data available, its effects on growth in particular probably cannot be detected.foreign aid, economic growth, data mining

An Index of Donor Performance

The Commitment to Development Index of the Center for Global Development rates 21 rich countries on the “development-friendliness” of their policies. It is revised and updated annually. In the 2004 edition, the component on foreign assistance combines quantitative and qualitative measures of official aid, and of fiscal policies that support private charitable giving. The quantitative measure uses a net transfers con- cept, as distinct from the net flows concept in the net Official Development Assistance measure of the Development Assistance Committee, which does not net out interest received. The qualitative factors are three: a penalty for tying aid; a discounting system that favors aid to poorer, better-governed recipients; and a penalty for “project proliferation.” The selectivity weighting approach avoids some conceptual problems inherent in the Dollar and Levin (2004) elasticity- based method. The proliferation pen-alty derives from a calibrated model of aid transaction cost developed in Roodman (forthcoming). The charitable giving measure is based on an estimate of the share of observed private giving to developing countries that is attributable to a) lower overall taxes (income effect) and b) specific tax incentives for giving (price effect). Despite the adjustments, overall results are dominated by differences in quantity of official aid given. This is because while there is a seven-fold range in net concessional transfers/GDP among the score countries, variation in overall aid quality across donors appears far lower, and private giving is generally small. Denmark, the Netherlands, Norway, and Sweden score highest while the largest donors in absolute terms, the United States and Japan, score in the bottom third. Standings by the 2004 methodology have been relatively stable since 1995.foreign aid, selectivity, performance measurement

Competitive Proliferation of Aid Projects: A Model

The proliferation of aid projects may overburden recipient governments with reporting requirements, donor visits, and other administrative overhead, siphoning off scarce domestic recipient resources, such as tax revenue or the time of skilled government officials, from directly productive use. But greater oversight may also improve the administration of projects, increasing development. I present a model of aid projects that reflects both sides of this coin. It posits a distinction between national-level governance and project-level governance. A donor can raise project-level governance above the baseline national level by requiring oversight activities of the recipient, although the benefits from doing so are less where national-level governance is already high. The model assumes that larger projects demand proportionally less oversight activity from the recipient. Comparative statics analysis suggests that to maximize development, projects should be larger where aid volume is higher, to avoid overburdening recipient administrative capacity; where recipient resources are scarcer, for the same reason; and where national governance is good, since the marginal benefit of oversight is then lower. A multi-donor generalization shows how donors that are imperfectly altruistic, caring most about the success of their own projects, will tend to sink into competitive proliferation, in which each donor subdivides its aid budget into smaller projects to raise the marginal productivity of the recipient’s resources in those projects and attract them away from other donors. The inefficiency arises from the lack of a market among donors for recipient resources. In a Nash equilibrium, competitive proliferation reduces overall development. But the smallest (selfish) donors can gain. This would discourage them from cooperating with other donors to contain competitive proliferation.Foreign aid, donor coordination, project proliferation

The Impact of Microcredit on the Poor in Bangladesh: Revisiting the Evidence

The most-noted studies on the impact of microcredit on households are based on a survey fielded in Bangladesh in the 1990s. Contradictions among them have produced lasting controversy and confusion. Pitt and Khandker (PK, 1998) apply a quasi-experimental design to 1991–92 data; they conclude that microcredit raises household consumption, especially when lent to women. Khandker (2005) applies panel methods using a 1999 resurvey; he concurs and extrapolates to conclude that microcredit helps the extremely poor even more than the moderately poor. But using simpler estimators than PK, Morduch (1999) finds no impact on the level of consumption in the 1991–92 data, even as he questions PK’s identifying assumptions. He does find evidence that microcredit reduces consumption volatility. Partly because of the sophistication of PK’s Maximum Likelihood estimator, the conflicting results were never directly confronted and reconciled. We end the impasse. A replication exercise shows that all these studies’ evidence for impact is weak. As for PK’s headline results, we obtain opposite signs. But we do not conclude that lending to women does harm. Rather, all three studies appear to fail in expunging endogeneity. We conclude that for non-experimental methods to retain a place in the program evaluator’s portfolio, the quality of the claimed natural experiments must be high and demonstrated.microcredit; impact evaluation; Grameen Bank; Bangladesh; replication; mixed-process models